Multiple transform utilization and applications for secure digital watermarking

ABSTRACT

Multiple transform utilization and applications for secure digital watermarking. In one embodiment of the present invention, digital blocks in digital information to be protected are transformed into the frequency domain using a fast Fourier transform. A plurality of frequencies and associated amplitudes are identified for each of the transformed digital blocks and a subset of the identified amplitudes is selected for each of the digital blocks using a primary mask from a key. Message information is selected from a message using a transformation table generated with a convolution mask. The chosen message information is encoded into each of the transformed digital blocks by altering the selected amplitudes based on the selected message information.

This application is a divisional of U.S. patent application Ser. No.09/053,628 filed Apr. 2, 1998, now U.S. Pat. No. 6,205,249, issued Mar.20, 2001.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. patent application Ser. No.08/587,943, filed Jan. 17, 1996, entitled “Method for Stega-CipherProtection of Computer Code,” the entire disclosure of which is herebyincorporated by reference U.S. Pat. No. 5,745,569.

FIELD OF THE INVENTION

The invention relates to the protection of digital information. Moreparticularly, the invention relates to multiple transform utilizationand applications for secure digital watermarking.

BACKGROUND OF THE INVENTION

Increasingly, commercially valuable information is being created andstored in “digital” form. For example, music, photographs and video canall be stored and transmitted as a series of numbers, such as 1's and0's. Digital techniques let the original information be recreated in avery accurate manner. Unfortunately, digital techniques also let theinformation be easily copied without the owner's permission.

Digital watermarks exist at a convergence point where creators andpublishers of digitized multimedia content demand local, secureidentification and authentication of content. Because piracy discouragesthe distribution of valuable digital information, establishingresponsibility for copies and derivative copies of such works isimportant. The goal of a digital watermark system is to insert a giveninformation signal or signals in such a manner as to leave little or noartifacts, with one standard being perceptibility, in the underlyingcontent signal, while maximizing its encoding level and “locationsensitivity” in the signal to force damage to the content signal whenremoval is attempted. In considering the various forms of multimediacontent, whether “master,” stereo, National Television StandardsCommittee (NTSC) video, audio tape or compact disc, tolerance of qualitywill vary with individuals and affect the underlying commercial andaesthetic value of the content. It is desirable to tie copyrights,ownership rights, purchaser information or some combination of these andrelated data into the content in such a manner that the contentundergoes damage, and therefore reduction of its value, with subsequentunauthorized distribution, commercial or otherwise. Digital watermarksaddress many of these concerns and research in the field has provided arich basis for extremely robust and secure implementations.

Of particular concern is the balance between the value of a digitized“piece” of content and the cost of providing worthwhile “protection” ofthat content. In a parallel to real world economic behavior, theperceived security of a commercial bank does not cause people toimmediately deposit cash because of the expense and time required toperform a bank deposit. For most individuals, possession of a US $100bill does not require any protection beyond putting it into a wallet.The existence of the World Wide Web, or “Web,” does not implicitlyindicate that value has been created for media which can be digitized,such as audio, still images and other media. The Web is simply a mediumfor information exchange, not a determinant for the commercial value ofcontent. The Web's use to exchange media does, however, provideinformation that helps determine this value, which is why responsibilityover digitized content is desirable. Note that digital watermarks are atool in this process, but they no not replace other mechanisms forestablishing more public issues of ownership, such as copyrights.Digital watermarks, for example, do not replace the “historical average”approach to value content. That is, a market of individuals willing tomake a purchase based solely on the perceived value of the content. Byway of example, a picture distributed over the Internet, or any otherelectronic exchange, does not necessarily increase the underlying valueof the picture, but the opportunity to reach a greater audience by thisform of “broadcast” may be a desirable mechanism to create “potentially”greater market-based valuations. That decision rests solely with therights holder in question.

Indeed, in many cases, depending on the time value of the content, valuemay actually be reduced if access is not properly controlled. With amagazine sold on a monthly basis, it is difficult to assess the value ofpictures in the magazine beyond the time the magazine is sold. Compactdisc valuations similarly have time-based variables, as well as tangiblevariables such as packaging versus the package-less electronic exchangeof the digitized audio signals. The Internet only provides a means tomore quickly reach consumers and does not replace the otherwise“market-based” value. Digital watermarks, properly implemented, add anecessary layer of ownership determination which will greatly assist indetermining and assessing value when they are “provably secure.” Thepresent invention improves digital watermarking technology whileoffering a means to properly “tamper proof” digitized content in amanner analogous to methods for establishing authenticity of real worldgoods.

A general weakness in digital watermark technology relates directly tothe way watermarks are implemented. Too many approaches leave detectionand decode control with the implementing party of the digital watermark,not the creator of the work to be protected. This fundamental aspect ofvarious watermark technologies removes proper economic incentives forimprovement of the technology when third parties successfully exploitthe implementation. One specific form of exploitation obscuressubsequent watermark detection. Others regard successful over encodingusing the same watermarking process at a subsequent time.

A set of secure digital watermark implementations address thisfundamental control issue, forming the basis of “key-based” approaches.These are covered by the following patents and pending applications, theentire disclosures of which are hereby incorporated by reference: U.S.Pat. No. 5,613,004 entitled “Steganographic Method and Device” and itsderivative U.S. patent application Ser. No. 08/775,216, U.S. patentapplication Ser. No. 08/587,944 entitled “Human Assisted Random KeyGeneration and Application for Digital Watermark System,” U.S. patentapplication Ser. No. 08/587,943 entitled “Method for Stega-CipherProtection of Computer Code,” U.S. patent application Ser. No.08/677,435 entitled “Optimization Methods for the Insertion, Protection,and Detection of Digital Watermarks in Digitized Data,” and U.S. patentapplication Ser. No. 08/772,222 entitled “Z-Transform Implementation ofDigital Watermarks.” Public key crypto-systems are described in U.S.Pat. Nos. 4,200,770, 4,218,582, 4,405,829 and 4,424,414, the entiredisclosures of which are also hereby incorporated by reference.

By way of improving these digital watermark security methods,utilization of multiple transforms, manipulation of signalcharacteristics and the requisite relationship to the mask set or “key”used for encoding and decoding operations are envisioned, as areoptimized combinations of these methods. While encoding a watermark mayultimately differ only slightly in terms of the transforms used in theencoding algorithm, the greater issues of an open, distributedarchitecture requires more robust approaches to survive attempts aterasure, or even means for making detection of the watermark impossible.These “attacks,” when computationally compared, may be diametricallyrelated. For instance, cropping and scaling differ in signal processingorientation, and can result in the weakening of a particularwatermarking approach but not all watermarking approaches.

Currently available approaches that encode using either a block-based orentire data set transform necessarily encode data in either the spatialor frequency domains, but never both domains. A simultaneous crop andscale affects the spatial and frequency domains enough to obscure mostavailable watermark systems. The ability to survive multiplemanipulations is an obvious benefit to those seeking to ensure thesecurity of their watermarked media. The present invention seeks toimprove on key-based approaches to watermarking previously disclosed,while offering greater control of the subsequently watermarked contentto rights owners and content creators.

Many currently available still image watermarking applications arefundamentally different from the key-based implementations. Suchproducts include products offered by Digimarc and Signum, which seek toprovide a robust watermark by encoding watermark messages that relyentirely on comparisons with the original image for decode operations.The subsequent result of the transform, a discrete cosine transformperformed in blocks, is digital signed. The embedded watermarks lack anyrelationship to the perceptual qualities of the image, making inverseapplication of the publicly available decoders a very good first line ofattack. Similarly, the encoding process may be applied by third parties,as demonstrated by some robustness tests, using one process to encodeover the result of an image watermarked with another process.Nonrepudiation of the watermark is not possible, because Digimarc andSignum act as the repository of all registrations of the image'sownership.

Another line of attack is a low pass filter that removes some of thehigh frequency noise that has been added, making error-free detectiondifficult or impossible. Finally, many tests of a simple JPEG transformindicate the watermarks may not survive as JPEG is based on the sametransforms as the encoding transforms used by the watermarking process.Other notable implementations, such as that offered by Signafy(developed by NEC researchers), appear to encode watermark messages byperforming a transform of the entire image. The goal of this process isto more consistently identify “candidate” watermark bits or regions ofthe image to encode in perceptually significant regions of the signal.Even so, Signafy relies on the original unwatermarked image toaccomplish decoding.

All of these methods still rely on the original unwatermarked image toensure relatively error-free detection of the watermarks. Thesteganographic method seeks to provide watermark security without anoriginal unwatermarked copy of the media for decode operations, as wellas providing users cryptographic security with ciphered symmetric keys.That is, the same key is used for encode and decode operations. Publickey pairs, where each user has a public/private key pair to performasymmetric encode and decode operations, can also be used. Discussionsof public key encryption and the benefits related to encryption are welldocumented. The growing availability of a public key infrastructure alsoindicates recognition of provable security. With such key-basedimplementations of watermarking, security can be off-loaded to the key,providing for a layered approach to security and authentication of thewatermark message as well as the watermarked content.

It is known that attacks on the survivability of other implementationsare readily available. Interesting network-based attacks on thewatermark message are also known which fool the central registrationserver into assuming an image is owned by someone other than theregistered owner. This also substantiates the concern that centralizedwatermarking technologies are not robust enough to provide properassurances as to the ownership of a given digitized copy of anmultimedia work.

Because the computational requirements of performing multiple transformsmay not be prohibitive for certain media types, such as still images andaudio, the present invention seeks to provide a means to securelywatermark media without the need for an original unwatermarked copy toperform decoding. These transforms may be performed in a manner notplainly evident to observers or the owner of the content, who may assumethe watermark is still detectable. Additionally, where a particularmedia type is commonly compressed (JPEG, MPEG, etc.), multipletransforms may be used to properly set the mask sets, prior to thewatermarking process, to alert a user to survivability prior to therelease of a watermarked, and thus perceived, “safe” copy to unknownparties. The result of the present invention is a more realisticapproach to watermarking taking the media type, as well as the provablesecurity of the keys into consideration. A more trusted model forelectronic commerce is therefore possible.

The creation of an optimized “envelope” for insertion of watermarks toestablish secured responsibility for digitally-sampled content providesthe basis of much watermark security but is also a complementary goal ofthe present invention. The predetermined or random key that is generatedis not only an essential map to access the hidden information signal,but is also the a subset of the original signal making directcomparisons with the original signal unnecessary. This increases theoverall security of the digital watermark.

Survival of simultaneous cropping and scaling is a difficult task withimage and audio watermarking, where such transformations are common withthe inadvertent use of images and audio, and with intentional attacks onthe watermark. The corresponding effects in audio are far more obvious,although watermarks which are strictly “frequency-based,” such asvariations of spread spectrum, suffer from alignment issues in audiosamples which have been “cropped,” or clipped from the original lengthof the piece. Scaling is far more noticeable to the human auditorysystem, though slight changes may affect frequency-only-type watermarkswhile not being apparent to a consumer. The far greater threat toavailable audio watermark applications, most of which are variations offrequency-based embedded signaling, are generally time-basedtransformations, including time-based compression and expansion of theaudio signal. Signafy is an example of spread spectrum-basedwatermarking, as are applications by Solana Technology, CRL, BBN, MIT,etc. “Spatial domain” approaches are more appropriate designations forthe technologies deployed by Digimarc, Signum, ARIS, Arbitron, etc.Interestingly, a time-based approached when considered for images isbasically a “spatial-based” approach. The pixels are “convolutional.”The difference being that the “spread spectrum-ed” area of thefrequencies is “too” well-defined and thus susceptible to over-encodingof random noise at the same sub-bands as that of the embedded signal.

Giovanni uses a block-based approach for the actual watermark. However,it is accompanied by image-recognition capable of restoring a scaledimage to its original scale. This “de-scaling” is applied before theimage is decoded. Other systems used a “differencing” of the originalimage with the watermarked image to “de-scale.” It is clear thatde-scaling is inherently important to the survival of any image, audioor video watermark. What is not clear is that the differencing operationis acceptable from a security standpoint. Moreover, differencing thatmust be carried out by the watermarking “authority,” instead of the useror creator of the image, causes the rights owner to lose control overthe original unwatermarked content. Aside from utilizing the mask setwithin the encoding/decoding key/key pair, the original signal must beused. The original is necessary to perform detection and decoding,although with the attacks described above it is not possible to clearlyestablish ownership over the watermarked content.

In view of the foregoing, it can be appreciated that a substantial needexists for multiple transform utilization and applications for securedigital watermarking that solve the problems discussed above.

SUMMARY OF THE INVENTION

The disadvantages of the art are alleviated to a great extent bymultiple transform utilization and applications for secure digitalwatermarking. In one embodiment of the present invention, digital blocksin digital information to be protected are transformed into thefrequency domain using a fast Fourier transform. A plurality offrequencies and associated amplitudes are identified for each of thetransformed digital blocks and a subset of the identified amplitudes isselected for each of the digital blocks using a primary mask from a key.Message information is selected from a message using a transformationtable generated with a convolution mask. The chosen message informationis encoded into each of the transformed digital blocks by altering theselected amplitudes based on the selected message information.

With these and other advantages and features of the invention that willbecome hereinafter apparent, the nature of the invention may be moreclearly understood by reference to the following detailed description ofthe invention, the appended claims and to the several drawings attachedherein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block flow diagram of a method for encoding digitalinformation according to an embodiment of the present invention.

FIG. 2 is a block flow diagram of a method for descaling digitalinformation according to an embodiment of the present invention.

FIG. 3 is a block flow diagram of a method for decoding digitalinformation according to an embodiment of the present invention.

DETAILED DESCRIPTION

In accordance with an embodiment of the present invention, multipletransforms are used with respect to secure digital watermarking. Thereare two approaches to watermarking using frequency-domain or spatialdomain transformations: using small blocks or using the entire data-set.For time-based media, such as audio or video, it is only practical towork in small pieces, since the entire file can be many megabytes insize. For still images, however, the files are usually much smaller andcan be transformed in a single operation. The two approaches each havetheir own strengths. Block-based methods are resistant to cropping.Cropping is the cutting out or removal of portions of the signal. Sincethe data is stored in small pieces, a crop merely means the loss of afew pieces. As long as enough blocks remain to decode a single, completewatermark, the crop does not remove the mark. Block-based systems,however, are susceptible to scaling. Scaling, such as affine scaling or“shrinking,” leads to a loss of the high frequencies of the signal. Ifthe block size is 32 samples and the data is scaled by 200%, therelevant data now covers 64 samples. However, the decoder still thinksthat the data is in 32 samples, and therefore only uses half the spacenecessary to properly read the watermark. Whole-set approaches have theopposite behavior. They are very good at surviving scaling, since theyapproach the data as a whole, and generally scale the data to aparticular size before encoding. Even a small crop, however, can throwoff the alignment of the transform and obscure the watermark.

With the present invention, and by incorporation of previously disclosedmaterial, it is now possible to authenticate an image or song or videowith the encoding key/key pair, eliminating false positive matches withcryptography and providing for the communication of a copyright throughregistration with third party authorities, instead of the originalunwatermarked copy.

The present invention provides an obvious improvement over the prior artwhile improving on previous disclosures by offsetting coordinate valuesof the original signal onto the key, which are then subsequently used toperform decode or detection operations by the user or authorized“key-holder.” This offsetting is necessary with content which may have awatermark “payload,” the amount of data that may successfully beencoded, based on Shannon's noisy channel coding theorem, that preventsenough invisible “saturation” of the signal with watermark messages toafford the owner the ability to detect a single message. An example, itis entirely possible that some images may only have enough of a payloadto carry a single 100 bit message, or 12 ASCII characters. In audioimplementations tested by the present inventor, 1000 bits per second areinaudibly encoded in a 16 bit 44.1 kHz audio signal. Most electronicallyavailable images do not have enough data to afford similar “payload”rates. Thus the premise that simultaneous cropping and scaling survivalis more difficult for images than a comparable commercially availableaudio or video track. The added security benefit is that the morelimited randomizer of a watermarking system based on spread spectrum orfrequency-only applications, the random value of the watermark data“hopping ”over a limited signaling band, is that the key is also anindependent source of ciphered or random data used to more effectivelyencode in a random manner. The key may actually have random valueslarger than the watermark message itself, measured in bits. Thewatermark decoder is assured that the image is in its original scale,and can decide whether it has been cropped based on its “de-scaled”dimensions.

The benefits of a system requiring keys for watermarking content andvalidating the distribution of said content is obvious. Different keysmay be used to encode different information while secure one way hashfunctions, digital signatures, or even one-time pads may be incorporatedin the key to secure the embedded signal and afford nonrepudiation andvalidation of the watermarked image and “its” key/key pair.Subsequently, these same keys may be used to later validate the embeddeddigital signature only, or fully decode the digital watermark message.Publishers can easily stipulate that content not only be digitallywatermarked, but that distributors must check the validity of thewatermarks by performing digital signature checks with keys that tackany other functionality.

Some discussion of secure digital watermarking has begun to appear.Leighton describes a means to prevent collusion attacks in digitalwatermarks in U.S. Pat. No. 5,664,018. Leighton, however, may notactually provide the security described. For example, in particularlyinstances where the watermarking technique is linear, the “insertionenvelope” or “watermarking space” is well-defined and thus susceptibleto attacks less sophisticated than collusion by unauthorized parties.Over encoding at the watermarking encoding level is but one simpleattack in such linear implementations. Another consideration ignored byLeighton is that commercially-valuable content in many cases may alreadyexist in a unwatermarked form somewhere, easily accessible to potentialpirates, gutting the need for any type of collusive activity. Suchexamples as compact disc or digitally broadcast video abound. Digitallysigning the embedded signal with preprocessing of watermark data is morelikely to prevent successful collusion. Depending on the media to bewatermarked, highly granular watermarking algorithms are far more likelyto successfully encode at a level below anything observable givenquantization artifacts, common in all digitally-sampled media, thanexpectations that a baseline watermark has any functionality.

Furthermore, a “baseline” watermark as disclosed is quite subjective. Itis simply described elsewhere in the art as the “perceptuallysignificant” regions of a signal: so making a watermarking function lesslinear or inverting the insertion of watermarks would seem to providethe same benefit without the additional work required to create a“baseline” watermark. Indeed, watermarking algorithms should already becapable of defining a target insertion envelope or region withoutadditional steps. Further, earlier disclosed applications by the presentinvention's inventor describe watermarking techniques that can be set toencode fewer bits than the available watermarking region's “bit-space”or encoding unrelated random noise in addition to watermark data toconfuse possible collusive or other attempts at erasure. The region of“candidate bits” can be defined by any number of compression schemes ortransformations, and the need to encode all of the bits is simplyunnecessary. What is evident is that Leighton does not allow for initialprevention of attacks on an embedded watermark as the content is visiblyor audibly unchanged. Moreover, encoding all of the bits may actuallyact as a security weakness to those who can replicate the regions with aknowledge of the encoding scheme. Again, security must also be offsetoutside of the actual watermark message to provide a truly robust andsecure watermark implementation.

In contrast, the present invention may be implemented with a variety ofcryptographic protocols to increase both confidence and security in theunderlying system. A predetermined key is described as a set of masks.These masks may include primary, convolution and message delimiters butmay extend into additional domains such as digital signatures of themessage. In previous disclosures, the functionality of these masks isdefined solely for mapping. Public and private keys may be used as keypairs to further increase the unlikeliness that a key may becompromised. Prior to encoding, the masks described above are generatedby a cryptographically secure random generation process. A block cipher,such as DES, in combination with a sufficiently random seed valueemulates a cryptographically secure random bit generator. These keyswill be saved along with information matching them to the sample streamin question in a database for use in descrambling and subsequentdetection or decode operation.

These same cryptographic protocols can be combined with embodiments ofthe present invention in administering streamed content that requiresauthorized keys to correctly display or play said streamed content in anunscrambled manner. As with digital watermarking, symmetric orasymmetric public key pairs may be used in a variety of implementations.Additionally, the need for certification authorities to maintainauthentic key-pairs becomes a consideration for greater security beyondsymmetric key implementations, where transmission security is a concern.

The following describes a sample embodiment of a system that protectsdigital information according to the present invention. Referring now indetail to the drawings wherein like parts are designated by likereference numerals throughout, there is illustrated in FIG. 1 a blockflow diagram of a method for encoding digital information according toan embodiment of the present invention. An image is processed by“blocks,” each block being, for example, a 32×32 pixel region in asingle color channel. At step 110, each block is transformed into thefrequency domain using a spectral transform or a Fast Fourier Transform(FFT). The largest 32 amplitudes are identified and a subset of these 32are selected using the primary mask from the key at steps 120 and 130.One message bit is then encoded into each block at steps 140 and 150.The bit is chosen from the message using a transformation tablegenerated using the convolution mask. If the bit is true, the selectedamplitudes are reduced by a user defined strength fraction. If the bitis false, the amplitudes are unchanged.

Each of the selected amplitudes and frequencies are stored in the key.After all of the image has been processed, a diagonal stripe of pixelsis saved in the key. This stripe can, for example, start in the upperleft corner and proceed at a 45 degree angle through the image. Theoriginal dimensions of the image are also stored in the key.

FIG. 2 is a block flow diagram of a method for descaling digitalinformation according to an embodiment of the present invention. When animage is chosen to be decoded, it first is checked to determine if ithas been cropped and/or scaled. If so, the image is scaled to theoriginal dimensions at step 210. The resulting “stripe,” or diagonalline of pixels, is fit against the stripe stored in the key at step 220.If the fit is better than the previous best fit, the scale is saved atsteps 230 and 240. If desired, the image can be padded with, forexample, a single row or column of zero pixels at step 260 and theprocess can be repeated to see if the fit improves.

If a perfect fit is found at step 250, the process concludes. If noperfect fit is found, the process continues up to a crop “radius” set bythe user. For example, if the crop radius is 4 the image can be paddedup to 4 rows and/or 4 columns. The best fit is chosen and the image isrestored to its original dimension, with any cropped area replaced byzeroes.

Once the in formation has been descaled, it can be decoded according toan embodiment of the present invention shown in FIG. 3. Decoding is theinverse process of encoding. The decoded amplitudes are compared withthe ones stored in the key in order to determine the position of theencoded bit at steps 310 and 320. The message is assembled using thereverse transformation table at step 330. At step 340, the message isthen hashed and the hash is compared with the hash of the originalmessage. The original hash had been stored in the key during encoding.If the hashes match, the message is declared valid and presented to theuser at step 350.

Although various embodiments are specifically illustrated and describedherein, it will be appreciated that modifications and variations of thepresent invention are covered by the above teachings and within thepurview of the appended claims without departing from the spirit andintended scope of the invention. Moreover, similar operations have beenapplied to audio and video content for time-based manipulations of thesignal as well as amplitude and pitch operations. The ability to descaleor otherwise quickly determine differencing without use of theunwatermarked original is inherently important for secure digitalwatermarking. It is also necessary to ensure nonrepudiation and thirdpart authentication as digitized content is exchanged over networks.

1. A method for descaling a digital image using a key, comprising thesteps of: determining original dimensions of the digital image from thekey; scaling the digital image to the original dimensions; obtaining areference subset of pixels from the key; and comparing the referencesubset of pixels with corresponding pixels in the scaled digital image.2. The method of claim 1, wherein said step of comparing determines afirst fit value based on the comparison, and wherein the method furthercomprises the steps of: padding the scaled digital image with an area ofpad pixels; and re-comparing the reference subset of pixels withcorresponding pixels in the padded image to determine a second fitvalue.
 3. The method of claim 2, wherein the area of pad pixels is a rowof single pixels.
 4. The method of claim 2, wherein the area of padpixels is a column of single pixels.
 5. The method of claim 2, whereinsaid steps of padding and re-comparing are performed a predeterminednumber of times.
 6. The method of claim 2, further comprising the stepof choosing a best fit value among the determined fit values andrestoring the digital image to the original size, including any padpixels associated with the best fit value.
 7. A method for descalingaudio digital information using a key, comprising the steps of:determining original dimensions of the audio digital information fromthe key; scaling the audio digital information to the originaldimensions; obtaining a reference subset of audio information from thekey; and comparing the reference subset of audio information withcorresponding information in the scaled audio information.
 8. A methodfor descaling a digital signal using a key, comprising the steps of:determining original dimensions of the digital signal from the key;scaling the digital signal to the original dimensions; obtaining areference signal portion from the key; and comparing the referencesignal portion with a corresponding signal portion in the scaled signal.